Codecov Report

❌ Patch coverage is 89.13043% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.66%. Comparing base (4da28b0) to head (b23310e).
⚠️ Report is 40 commits behind head on main.

@@            Coverage Diff             @@
##             main     #708      +/-   ##
==========================================
- Coverage   93.67%   93.66%   -0.02%     
==========================================
  Files          89       89              
  Lines       13621    13707      +86     
==========================================
+ Hits        12759    12838      +79     
- Misses        862      869       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

In your original description you talked about the performance of _verify_input_object_type. How does this PR affect that? Where is this function?

Oh and what was your test case for your timing? I'm surprised by how fast the query stuff was, but even more by how slow the NaN case was for the radius of influence stuff.

One last thing: I'm not ignoring your other pyresample and satpy PRs, but I'm still catching up on real work after being out for a couple days and I'm trying to fully understand the changes before commenting. Thanks again for all these changes/fixes/improvements.

In your original description you talked about the performance of _verify_input_object_type. How does this PR affect that? Where is this function?

pyresample/future/resamplers/nearest.py:496 replaces data.copy() with data.copy(deep=False) before swapping .data to a dask array. That avoids a deep copy of the underlying NumPy buffer.

Oh and what was your test case for your timing? I'm surprised by how fast the query stuff was, but even more by how slow the NaN case was for the radius of influence stuff.

I don't remember now, but I was measuring the specific code paths in isolation, not in a bigger system. I like efficient software.

One last thing: I'm not ignoring your other pyresample and satpy PRs, but I'm still catching up on real work after being out for a couple days and I'm trying to fully understand the changes before commenting. Thanks again for all these changes/fixes/improvements.

Np, I was delaying too because I started working on the data I got processed 😄.

I don't remember now, but I was measuring the specific code paths in isolation, not in a bigger system. I like efficient software.

Were you running things multiple times? I'm unable to get my laptop with a real world case to perform in a consistently better way. I'll try a few more things to sanity check...

I don't remember what data you said you're processing, but I'm taking ABI L1b data (G16) and resampling it using nearest neighbor to the Satpy builtin "brazil" area. With current pyresample main I first got 11.5s so I switched to this branch and got 10.87s so I thought "awesome, faster". Then I ran it over and over again and got numbers from 11.63s to 11.86s. Then I went back to main and got 11.11s to 11.83s.

from datetime import datetime
from satpy import Scene
from glob import glob

scn = Scene(reader="abi_l1b", filenames=glob("/data/satellite/abi/20180623/OR_ABI-L1b-RadF-M3C*G16*.nc"))
scn.load(["C02"])

first = datetime.now()
new_scn = scn.resample("brazil", resampler="nearest")
new_scn["C02"].compute()
#new_scn.save_datasets(filename="C02_brazil.tif")
second = datetime.now()

print((second - first).total_seconds())

In my PR body, I performed 3 distinct measurements on the individual methods directly, not the full pipeline. I do confirm that the improvements are still reproducible. I've also noticed I implemented one more valuable specialization to query_no_distance that brings 6% performance improvement in the neighbors=1 common case.

Thanks. I've been off of work the last week or so but am catching up on things today. I'll try to review this when I get a chance. And yeah if you're testing the individual functions it makes sense to see these low-level improvements but then I not see them on the high-level when the majority of time is spent building the KDTree and the actual query time.